After integrating this some more with tensorstore, I've reworked the library lookup to handle manylinux and other linux installs with no dynamic libraries exposed.

rickeylev edit to preserve some initial PR info:

Example get_local_runtime_info.py outputs:

rules_python:local_runtime_repo(@@local_python3) INFO: GetPythonInfo result:
  INSTSONAME: Python.framework/Versions/3.12/Python
  LDLIBRARY: Python.framework/Versions/3.12/Python
  LIBDIR: /Library/Frameworks/Python.framework/Versions/3.12/lib
  MULTIARCH: darwin
  PY3LIBRARY: <empty string>
  PYTHONFRAMEWORKPREFIX: /Library/Frameworks
  base_executable: /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.12
  implementation_name: cpython
  include: /Library/Frameworks/Python.framework/Versions/3.12/include/python3.12
  major: 3
  micro: 10
  minor: 12

Here we see the MacOS framework issue, namely LDLIBRARY is not under LIBDIR, but directly under frameworks.
The new code adds PYTHONFRAMEWORKPREFIX to the search path, so the library is found.

rules_python:local_runtime_repo(@@local_python3)  INFO: GetPythonInfo result:
  INSTSONAME: None
  LDLIBRARY: python313.dll
  LIBDIR: T:\build_temp\home\python_4379bdec28bdff81a567a01c9b8cf10e3856c8c966e4fe53945bedea6338b416\tools\libs
  MULTIARCH: None
  PY3LIBRARY: python313.lib
  base_executable: T:\build_temp\home\python_4379bdec28bdff81a567a01c9b8cf10e3856c8c966e4fe53945bedea6338b416\tools\python.exe
  implementation_name: cpython
  include: T:\build_temp\home\python_4379bdec28bdff81a567a01c9b8cf10e3856c8c966e4fe53945bedea6338b416\tools\Include
  major: 3
  micro: 1
  minor: 13

Here we see that under windows the python313.dll is reported; python313.lib is added via PY3LIBRARY.

Three more notes on this CL; they may be areas for future improvements.

• On MacOS, when linking against the system python, we may want to use "-Wl,-framework,Python". However this CL does not detect / change those linker options.

• It would be nice to have tests for a minimal C python extension and minimal C python embedded use for the local_runtime as well as the hermetic runtimes; this is similar to tests/cc/current_cc_py_libs

Oddly, I can edit and commit files via the ui, but can't push to laramiel/main on the command line. Is there something special setup with your repo or branch? I was going to move the example over into local toolchains, but it was a bit hard to refactor that via the ui alone.

I wouldn't expect so; I forked using the web ui and haven't changed anything.

Oddly, I can edit and commit files via the ui, but can't push to laramiel/main on the command line. Is there something special setup with your repo or branch? I was going to move the example over into local toolchains, but it was a bit hard to refactor that via the ui alone.

I wouldn't expect so; I forked using the web ui and haven't changed anything.

Weird 🤷 . Maybe because your fork's branch is the main branch? sometimes those have additional protections.

Looks like we have enough slots to run the CI jobs, but they had a lot of unrelated failures. On the plus side, I think they exercised enough of the new local toolchain logic to give some basic validation of the new code.

I don't think we should block this PR on making all of those integration tests pass. I'll reconfig the mac and windows ones to just run the local toolchain test; that looks relatively easy and feasible.

re: adding a py extension test: +1 on such a test (we have need of that elsewhere for testing), but I'm also OK with omitting it for this PR. Test-supporting code goes in tests/support

I guess py_extension is being added for a test now but might later be made into a public rule?

For fvisibility=hidden, really that needs to propagate to all transitive dependencies instead, so should be handled via a transition or just by a global --copt option specified by the user.

I guess py_extension is being added for a test now but might later be made into a public rule?

It's added for test purposes, I'll leave it up to @rickeylev or others to make that determination.
It would be nice to have a common py_extension, IMO.

For fvisibility=hidden, really that needs to propagate to all transitive dependencies instead, so should be handled via a transition or just by a global --copt option specified by the user.

For now I've removed the cc_library from py_extension. That seems a little cleaner?

I guess py_extension is being added for a test now but might later be made into a public rule?

Maybe, but not without some more consideration and evaluation. (e.g. My current day-job team developed their own variation of py_extension, my last team created another one, and I've seen others out there, all with pretty different implementations). #824 is the FR for this (long with lots of details and cases). Your mention of how deps are handled and fvisibility are good examples of the sort of considerations to not forget. I don't want perfect to be the enemy of the good, but also want thoughtfulness put into a public feature. (All that plus I want to keep the PR scope reasonable)